{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Welcome to the coding portion of the SimOpt workshop!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ignore some warnings that pop up since this is running in a Jupyter notebook\n",
    "# ruff: noqa: E402, F811\n",
    "\n",
    "# Some setup...\n",
    "\n",
    "import os\n",
    "\n",
    "os.chdir(\"../\")  # Move one level up to import simopt\n",
    "\n",
    "import sys\n",
    "\n",
    "sys.path.append(\"venv\\\\lib\\\\site-packages\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CODE CELL [0]\n",
    "\n",
    "# Import experiment_base module, which contains functions for experimentation.\n",
    "import simopt.experiment_base as expbase\n",
    "\n",
    "# Import Example problem and Random Search and ADAM solvers.\n",
    "from simopt.models.example import ExampleProblem\n",
    "from simopt.solvers.randomsearch import RandomSearch\n",
    "from simopt.solvers.adam import ADAM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this portion of the workshop, we'll be working with a problem and two solvers.\n",
    "\n",
    "**Problem:** Minimize $||x||^2$ with additive Gaussian noise over $x \\in \\mathbb{R}^2$.\n",
    "\n",
    "**Solver:** Random Search\n",
    "* Randomly samples solutions. For this two-dimensional problem, solutions are sampled from a MVN distribution with mean vector (0, 0) and variance-covariance matrix (1, 0; 0, 1).\n",
    "* Takes a fixed number of observations (replications) at each solution.\n",
    "* [Full documentation](https://simopt.readthedocs.io/en/latest/randomsearch.html)\n",
    "\n",
    "**Solver:** ADAM\n",
    "* A gradient-based search. Direct (IPA) gradient estimators are used, if available. Otherwise a finite differences estimator is used.\n",
    "* Takes a fixed number of observations (replications) at each solution. This parameter is called `r`.\n",
    "* [Full documentation](https://simopt.readthedocs.io/en/latest/adam.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CODE CELL [1]\n",
    "\n",
    "# Instantiate the problem and the Random Search solver, with specifications.\n",
    "my_problem = ExampleProblem(\n",
    "    fixed_factors={\"initial_solution\": (2.0, 2.0), \"budget\": 200}\n",
    ")\n",
    "my_rand_search_solver = RandomSearch(\n",
    "    fixed_factors={\"crn_across_solns\": True, \"sample_size\": 10}\n",
    ")\n",
    "\n",
    "# Pair the problem and solver for experimentation.\n",
    "myexperiment = expbase.ProblemSolver(\n",
    "    problem=my_problem, solver=my_rand_search_solver\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how Random Search does on this toy problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CODE CELL [2]\n",
    "\n",
    "# Run 10 macroreplications of Random Search on the Example Problem.\n",
    "myexperiment.run(n_macroreps=10)\n",
    "\n",
    "# Post-process the results.\n",
    "myexperiment.post_replicate(n_postreps=200)\n",
    "expbase.post_normalize(experiments=[myexperiment], n_postreps_init_opt=200)\n",
    "\n",
    "# [Results are saved in a file called experiments/outputs/RNDSRCH_on_EXAMPLE-1.pickle.]\n",
    "# [The file is not human-readable, so we'll skip looking at it.]\n",
    "\n",
    "# Record a summary of the results in a human-readable way.\n",
    "myexperiment.log_experiment_results()\n",
    "# [Go check out the file called experiments/logs/RNDSRCH_on_EXAMPLE-1_experiment_results.txt.]\n",
    "\n",
    "# Plot the (unnormalized) progress curves from the 10 macroreplications.\n",
    "expbase.plot_progress_curves(\n",
    "    experiments=[myexperiment], plot_type=\"all\", normalize=False\n",
    ")\n",
    "# Plot the (unnormalized) mean progress curve with bootstrapped CIs.\n",
    "expbase.plot_progress_curves(\n",
    "    experiments=[myexperiment], plot_type=\"mean\", normalize=False\n",
    ")\n",
    "# [The plots should be displayed in the output produced below.]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Your turn.\n",
    "\n",
    "### Exercise \\#1\n",
    "\n",
    "In CODE CELL [1], play around with the arguments when initializing `myproblem` and `mysolver`.\n",
    "\n",
    "Vary factors of the Example problem:\n",
    "- Change the initial solution.\n",
    "- Change the budget, i.e., the max number of replications. \n",
    "\n",
    "Vary factors of the Random Search solver:\n",
    "- Change whether it uses CRN across solutions.\n",
    "- Change the number of replications it takes at each solution.\n",
    "\n",
    "Rerun CODE CELLS [1] and [2]. *What do you observe?*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Now let's work with the source code.\n",
    "\n",
    "### Exercise \\#2\n",
    "\n",
    "1. Open the file simopt/model/example.py in the VS Code editor.\n",
    "2. Let's change how random search randomly samples solutions in R^2. For starters, uncomment Line 430\n",
    "\n",
    "    `x = tuple([rand_sol_rng.uniform(-2, 2) for _ in range(self.dim)])`\n",
    "\n",
    "    and comment out Lines 431-437\n",
    "    \n",
    "    `x = tuple(rand_sol_rng.mvnormalvariate(mean_vec=np.zeros(self.dim), cov=np.eye(self.dim), factorized=False))`\n",
    "\n",
    "3. Restart the kernel using the Restart Button at the top of this notebook. This will ensure the new version of the source code is being imported.\n",
    "4. Run COMBO CODE CELL [0 + 1 + 2] below (this effectively reruns CODE CELLS [0], [1], and [2]). *How have the plots changed?*\n",
    "\n",
    "**Extra for Experts:** Come up with your own sampling distribution. Documentation on the types of distributions available can be found [here](https://mrg32k3a.readthedocs.io/en/latest/mrg32k3a.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# COMBO CODE CELL [0 + 1 + 2]\n",
    "import os\n",
    "\n",
    "os.chdir(\"../\")\n",
    "import sys\n",
    "\n",
    "sys.path.append(\"venv\\\\lib\\\\site-packages\")\n",
    "import simopt.experiment_base as expbase\n",
    "from simopt.models.example import ExampleProblem\n",
    "from simopt.solvers.randomsearch import RandomSearch\n",
    "from simopt.solvers.adam import ADAM\n",
    "\n",
    "my_problem = ExampleProblem(\n",
    "    fixed_factors={\"initial_solution\": (2.0, 2.0), \"budget\": 200}\n",
    ")\n",
    "my_rand_search_solver = RandomSearch(\n",
    "    fixed_factors={\"crn_across_solns\": True, \"sample_size\": 10}\n",
    ")\n",
    "myexperiment = expbase.ProblemSolver(\n",
    "    problem=my_problem, solver=my_rand_search_solver\n",
    ")\n",
    "\n",
    "myexperiment.run(n_macroreps=10)\n",
    "myexperiment.post_replicate(n_postreps=200)\n",
    "expbase.post_normalize(experiments=[myexperiment], n_postreps_init_opt=200)\n",
    "myexperiment.log_experiment_results()\n",
    "expbase.plot_progress_curves(\n",
    "    experiments=[myexperiment], plot_type=\"all\", normalize=False\n",
    ")\n",
    "expbase.plot_progress_curves(\n",
    "    experiments=[myexperiment], plot_type=\"mean\", normalize=False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Now let's bring the ADAM solver into the mix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CODE CELL [3]\n",
    "\n",
    "my_adam_solver = ADAM(fixed_factors={\"crn_across_solns\": True, \"r\": 10})\n",
    "# Create a grouping of Example-RandomSearch and Example-ADAM pairs.\n",
    "mygroupexperiment = expbase.ProblemsSolvers(\n",
    "    problems=[my_problem], solvers=[my_rand_search_solver, my_adam_solver]\n",
    ")\n",
    "\n",
    "# Run 10 macroreplications of each pair and post-process.\n",
    "mygroupexperiment.run(n_macroreps=10)\n",
    "mygroupexperiment.post_replicate(n_postreps=200)\n",
    "mygroupexperiment.post_normalize(n_postreps_init_opt=200)\n",
    "\n",
    "# Record a summary of the results in a human-readable way.\n",
    "mygroupexperiment.log_group_experiment_results()\n",
    "# [Go check out the file called experiments/logs/group_RNDSRCH_ADAM_on_EXAMPLE-1_group_experiment_results.txt.]\n",
    "\n",
    "# Plot the mean progress curve for each solver from the 10 macroreplications.\n",
    "expbase.plot_progress_curves(\n",
    "    experiments=[\n",
    "        mygroupexperiment.experiments[0][0],\n",
    "        mygroupexperiment.experiments[1][0],\n",
    "    ],\n",
    "    plot_type=\"mean\",\n",
    "    normalize=False,\n",
    ")\n",
    "# [The plot should be displayed in the output produced below.]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "#### Your turn.\n",
    "\n",
    "### Exercise \\#3\n",
    "\n",
    "1. Open simopt/model/example.py again.\n",
    "2. Change the noise in the objective function evaluations to create a slightly different 2D optimization problem. This can be done by changing Line 99: \n",
    "    \n",
    "    `fn_eval_at_x = np.linalg.norm(x) ** 2 + noise_rng.normalvariate()`\n",
    "\n",
    "    where `x` is a numpy array of length two. For starters, try passing the argument `sigma=10` into the function call `noise_rng.normalvariate()`. The default value is `sigma=1`, so this has the effect of increasing the common variance of the noise from 1 to 100.\n",
    "3. Restart the kernel and run COMBO CODE CELL [0 + 1 + 3] below. *How have the plots changed? Why haven't they changed more?*\n",
    "\n",
    "4. Next, change the underlying objective function by replacing `np.linalg.norm(x) ** 2` in Line 99 with some other two-dimensional function of `x`, e.g., `1 - np.exp(-np.linalg.norm(x) ** 2)`. (This objective function looks like an upside-down standard bivariate normal pdf, rescaled.)\n",
    "5. Depending of your choice of new objective function, you MAY need to change other parts of the code, including:\n",
    "    * The gradient of `f(x)` in Line 103. For the example given above, this would need to be changed from\n",
    "    \n",
    "        `gradients = {\"est_f(x)\": {\"x\": tuple(2 * x)}}`\n",
    "\n",
    "        to\n",
    "\n",
    "        `gradients = {\"est_f(x)\": {\"x\": tuple(2 * x * np.exp(-np.linalg.norm(x) ** 2))}}`\n",
    "    * If you change the problem to a maxmization problem, you will need to change Line 190 from\n",
    "    \n",
    "        `return (-1,)`\n",
    "        \n",
    "        to\n",
    "        \n",
    "        `return (1,)`.\n",
    "    * The optimal solution in Line 214. (For the running example, this will not be necessary.)\n",
    "    * The optimal objective function value in Line 208. (For the running example, this will not be necessary.)\n",
    "6. Restart the kernel and run COMBO CODE CELL [0 + 1 + 3] below. *How have the plots changed?*\n",
    "\n",
    "**Extra for Experts:** Change the dimension of the problem. To do this, you will need to change the dimension of the default initial solution, defined in Line 185."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# COMBO CODE CELL [0 + 1 + 3]\n",
    "import os\n",
    "\n",
    "os.chdir(\"../\")\n",
    "import sys\n",
    "\n",
    "sys.path.append(\"venv\\\\lib\\\\site-packages\")\n",
    "import simopt.experiment_base as expbase\n",
    "from simopt.models.example import ExampleProblem\n",
    "from simopt.solvers.randomsearch import RandomSearch\n",
    "from simopt.solvers.adam import ADAM\n",
    "\n",
    "my_problem = ExampleProblem(\n",
    "    fixed_factors={\"initial_solution\": (2.0, 2.0), \"budget\": 200}\n",
    ")\n",
    "my_rand_search_solver = RandomSearch(\n",
    "    fixed_factors={\"crn_across_solns\": True, \"sample_size\": 10}\n",
    ")\n",
    "my_adam_solver = ADAM(fixed_factors={\"crn_across_solns\": True, \"r\": 10})\n",
    "\n",
    "mygroupexperiment = expbase.ProblemsSolvers(\n",
    "    problems=[my_problem], solvers=[my_rand_search_solver, my_adam_solver]\n",
    ")\n",
    "mygroupexperiment.run(n_macroreps=10)\n",
    "mygroupexperiment.post_replicate(n_postreps=200)\n",
    "mygroupexperiment.post_normalize(n_postreps_init_opt=200)\n",
    "mygroupexperiment.log_group_experiment_results()\n",
    "expbase.plot_progress_curves(\n",
    "    experiments=[\n",
    "        mygroupexperiment.experiments[0][0],\n",
    "        mygroupexperiment.experiments[1][0],\n",
    "    ],\n",
    "    plot_type=\"mean\",\n",
    "    normalize=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Other demonstrations, time permitting\n",
    "* Walkthrough `replicate()` method of simopt/models/ironore.py to illustrate what the code looks like for a typical stochastic simulation model.\n",
    "* Walkthrough `solve()` method of simopt/solvers/ADAM.py to illustrate what the code looks like for a typical solver."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
